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MEraOD AND SYSTEM FOR GENERATING AND SEARCHING AN OFTIMAL 
MAXIMUM LIKELIHOOD DEOSION TREE FOR HIDDEN MARKOV MODEL 
(HMM) BASED SPEECH RECOGNITION 

FIELD OF THE INVENTION 

The present invention relates generally to speech processing aind to large 
vocabtilary continuotis speech recognition (LVCSR) systems. More particularly, the 
present invention relates to a method and system for generating and searching an 
optimal maximum likelihood decision tree (OML^DCSTree) for hidden markov model 
(HMM) based speech recognition. 

BACKGROUND OF THE INVENTION 

A speech recogrdtion system recognizes a rallection of spoken words ("speech") 
into recognized phrases or sentences. A spoken word tj^ically indudes one or more 
phones or phonemes, which are distinct sounds of a spoken word. Thus, to recognize 
speech, a speech recognition system must determine relationships between the words 
in the speech. A common way of determining relationships between words in 
recognizing speech is by using state decision tree clustering under a continuous 
density hidden markov model (HMM). 

Typically, a HMM tises a series of transitions from state to state to model a 
letter, a word, or a sentence. Each state can be referred to a phone or a triphone 
(phone having a left and right context). Each arc of the transitions has an associated 
probability, which gives die probability of the transition from one state to the next at 
an end of an observation frame. As such, an imknown speech aghal can be 
represented by ordered states with a given probability. Moreover, words in an 
unknown speech signal can be recognized by using the ordered states of the HMM. 

Currently, there are two popular approaches in building a decision tree-based 
HMM, which are a rule-based decision tree approach and a data driven approach. The 
rale-based decision tree approach uses a set of simple or composite phonetic context 
questions (questions requiring simple "yes"^ or "no" answers) as a set of rules to 
constract a HMM state decision tree according to a maximum likelihood (ML) 
principle. By way of contrast, the data driven approach groups triphones using 
specific training data obtained during a training process. 
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A disadvantage of the rule-based decision tree approach is that there is no 
fairness guaranty for each individual rule regardless if it is derived by a linguistic 
expert, a database, or by both. That is, for simple phonetic context question based 
rules, all triphones are grouped into two yes/no classes according to their phonetic 
contextual information. However, some of the triphones, contrary to the rule, may 
have a better maximiam likelihood (ML) criteria in another dass. The data driven 
approach can provide better guarantees for optimal ML dassiificatipn for known 
triphones (i-e., triphones that are obtained during the training process). A dis- 
advantage of the data driven approach, however, is that the tree built with the data 
driven approach has problems dealing with triphones tfiat did not appear during the 
training process, but do appear in the recognition process. Consequently, unknown 
triphones, which were not detected during the training process, may lower speech 
recognition adcizracy- 

BRIEF DESCRTFnON OF T HE DRAWINGS 

The features and advantages of flie present invention are iUtistrated by way of 
example and not intended to be limited by the figures of the accompanying drawings 
in which like references indicate similar elements, and in which: 

FIG. 1 is a diagram illustrating an exemplary digital processing system for 
implementing the present invention; 

FIG. 2 is a block diagram of a speech processing system according to one 
embodiment; 

FIG. 3 is a functional flow diagram illustrating an operation using an optimal 
maximum likelihood decision tree according to one einbodiment; 

FIG. 4 is a flow chart illustrating an operation to build an optimal maximum 
likelihood decision tree according to one embodimi^t; 

FIG. 5 is an exemplary diagram of tree nodes in a decision tree; 

FIG. 6 is an exemplary diagram of a niaximimi likelihood decision tree to 
illustrate decoding using the maximtim likelihood decision tree; and 

FIG. 7 is a flow chart illustrating an operation to decode speech using an 
maximimi likelihood tree according to one embodiment 
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DETAILED PESCRIFnON 

According to one aspect of the present invention a method is disclosed for 
generating and searching an optimal likelihood decision tree (OML-DSCTree) for 
hidden markov model (HMM) based speech recognition. For one embodiment, speech 
signals are received. The received speech signals are processed to generate a plurality 
of phoneme dusters. Tlie phonane dusters (e.g., biphone or triphone dusters) are 
grouped into a first duster node and a second duster node according to their answers 
to phonetic context questions. A determination is made if a phoneme duster in the 
first duster node is to be moved into the second duster node based on a likelihood 
increase of the phone duster of the first duster node from being in the first duster 
node to being in the second dtister node, 

• The following speech processing techniques described herein can provide 
higher speech recognition accuracy by reshuffling phoneme dusters between first 
node duster and a second node duster based on the likelihood increase. The following 
speech processing techniques can be used to build a decision tree, which can ensiire 
optimal maximum likelihood (ML). By controlling a ttireshold, the following speech 
processing techniques can be used to generate a decision tree with any combination of 
a rule-based approadi and a data driven approadi under a unified decision tree 
structure based on ML prindples. 

In the following description, an optimal maximum likelihood dedsion tree 
(OML-DCSTree) is described for speech recognition, however, the OML-DCSTree can 
also be used for pattern recognition applications. 

FIG. 1 is a diagram illustrating an exemplary digital processing system 100 for 
implementing the present invention. The speech processing and speech recognition 
tedmiques described herein can be implemented and utilized within digital processing 
system 100, which can represent a general purpose computer, portable computer, 
hand-held electronic device, or otiier like device. The components of digital 
processing system 100 are exemplary in which one or more components can be 
omitted or added. For exair^Ie, one or more memory devices can be utilized for 
digital processing system 100. 

Referring to FIG. 1, digital processing S5rstem 100 indudes a central processing 
unit 102 and a signal processor 103 coupled to a display circuit 105, main memory 104, 
static memory 106, and mass storage device 107 via bus 101. Digital processing system 
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100 can also be coupled to a display 121, keypad input 122, cursor control 123, hard 
copy device 124, input/output (I/O) devices 125, and audio/speech device 126 via 
bus 101. 

Bus 101 is a standard system bus for communicating information and signals. 
CPU 102 and signal processor 103 are processing imits for digital processing system 
100. CPU 102 or signal processor 103 or hotix can be used to process iziformation 
and/or signals for digital processing system 100. Signal processor 103 can be used to 
process speech or audio information and signals for speech processing and 
recognition. Alternatively, CPU 102 can be used to process speech or audio 
information and signals for speech processing or recognition. CPU 102 includes a 
control unit 131, an arithmetic logic unit (ALU) 132, and several registers 133, which 
are used to process information and signals. Signal processor 103 can also include 
similar components as CPU lOZ 

Main memory 104 can be, e.g., a random access memory (RAM) or some other 
dynamic storage device, for storing information or instructions (program code), which 
are used by CPU 102 or signal processor 103. For example, main memory 104 may 
store speech or audio ii:iformation and instructions to be executed by signal processor 
' 103 to process the speech or audio information. Main memory 104 may also store 
temporary variables or other intermediate information during execution of 
instructions by CPU 102 or signal processor 103. Static memory 106, can be, e.g., a 
read only memory (ROM) and/or other static storage devices, for storing information 
or instructionis, which can also be used by CPU 102 or signal processor 103. Mass 
storage device 107 can be, e.g., a hard or floppy disk drive or optical disk drive, for 
storing information or instructions for digital processing system 100. 

Display 121 can be, e.g., a cathode ray tube (CRT) or liquid aystal display (LCD). 
Display device 121 displays information or graphics to a user. Digital processing 
system 101 can interface with disphy 121 via dispkydrcuit 105. Keypad input 122 is a 
alphantimeric input device for communicating information and command selections to 
digital processing system 100. Cursor control 123 can be, e.g., a mouse, a trackball, or 
cursor direction keys, for controlling movement of an object on display 121. Hard 
copy device 124 can be, e.g., a laser printer, for printing information on paper, film, or 
some other like medium. A number of input/ output devices 125 can be coupled to 
digital processing system 100. For example, a speaka: can be coupled to digital 
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processing sj^em 100. Audio/ speech device 126 can be, e.g., a microphone with an 
analog to digital converter, for caphiring sounds of speech in an analog form and 
transforming tihe soxmds into digital form, wluch can be used by signal processor 203 
and/br CPU 102, for speech processing or recognition. 

The speech processing and speech recognition techniques described herein can be 
implemented by hardware and/or software contained within digital processing 
system 100. For example, CPU 102 or signal processor can execute code or instructions 
stored in a madiine-rjeadable medium, e.g., main memory 104, to process or to 
recognize speech. 

The machine-readable medium may indude a mechanism that provides (i.e., 
stores and/or transmits) information in a form readable by a machine such as 
computer or digital processing device. For example, a machine-readable medium may 
include a read only memory (ROM), random access memory (RAM), magnetic disk 
storage media, optical storage media, flash memory devices. The code or instructions 
can be represented by carrier wave signals, infrared signals, digital signals, and by 
other like signals. 

FIG. 2 is a block diagram of a speech processing system 200 according to one 
embodiment The ^eedi recognition systemi geui be implemented, e.g., in digital 
procesabg ^tem 100 as described in FIG. 1. Referring to FIG. 2, block diagram 200 
includes an audio/speech device 204 for receiving and processing speech signals 202 
and a signal processor 206 for processing speech signals 402 from audio/speech device 
204. For one embodiment, signal processor 206 can build an OML-DCSTree as 
described in FIG. 4 to be stored as acoustical models 208 in a training process. For 
another embodiment, signal processor 206 can use acoustical models 208 (e.g., an 
OML-DCSTree) to recognize speech during a speech recognition process. 

Audio/speech device 204 is an audio and speech receiving mechanism* For 
example, audio/speech device 204 can be a microphone. A user can speak into 
audio/speech device 203 in which acoustics (i.e., analog signals) are provided to 
audio/speech device 203. Audio/speech device 204 can convert the analog signals 
(i.e., speech signals 202) into digital form. 

Signal processor 206 is a prociessing device, which can be used during a training 
process or a continuoiis speech recognition process. During a training process, signal 
processor .206 builds relationships for flie words contained in speech signals 202 to 
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create acoustical models 208. For exan^le, acoustical models 208 can be based on a 
hidden markov model (HMM). During a speech recognition process, signal processor 
206 receives imknown speech signals 202. Signal processor searches the acoustical 
models 208 to match words in the imknown speech signals with known states in the 
acoustical models 208. If words in the imknown speech signals match known states of 
the acoxistical models 208^ processor 206 can output a recognized output such as a text 
file including known words, phrases, or sentaaces. 

FIG. 3 is a functional flow diagram illustrating an operation 300 using an optimal 
maximum decision likelihood (OML-DCSTree) according to one embodiment 
Referring to FIG. 3, at functional block 302, speech is inputted into a feature extractor* 
The feature extractor extracts data from the analog form of the speech. For example, 
the feature extractor can extract frequency data such as, for example, mal-frequency 
cepstrum coefficients (MFCQ and its first and second derivatives from the speech. 

At functional block 304, the data from the feature extractor can be either trained 
(for a training process) or recognized (for a recognition process). At functional blocks, 
306 and 308, if data from tiie feature extractor is to be trained, the data is trained into 
an OML-DCSTree based on a HMM. For example, the operation as explained in FIG. 4 
can be used to build the OMLrDCSTree. 

At functional block 310, if data from tire feature extractor is to be recognized, 
the data can be recognized using an OML-DCSTree HMM recognizer, which can use 
the OML-DCSTree based HMM or a pre-trained language model (LM) to generate 
recognized speech. For example, the operation as explained in FIG. 7 can be used to 
recognize speech using the OML-DCSTree. The LM component of the system depends 
on the task, which can be a popxilar N-gram model for a dictation-like system task or a 
finite-state-grammar model for a dialog task. 

HG. 4 is a flow chart illustrating an operation 400 to build an optimal maximum 
likelihood decision tree (OML-DCSTree) according to one embodiment. The following 
operation 400 is built on top of a phonetic question based state decision tree 
framework such as a rule-based approach framework. That is, for eadi question in a 
question set, all the triphones are first classified into two (yes/no) nodes. For purposes 
of explanation, operation 400 starts at operation 402. 

Referring to HG. 4, at operation 402, all the triphone clusters are collected that 
are to be clustered starting from a root node. At operation 404, cluster all the triphone 
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dusters into two chUdren nodes: Y-Node and N-Node according to their Y/N answers 
to the "best" question. 

At operation 406, a node is arbitrarily picked, e.g., a Y-Node and the operation 
is to find one cluster (Yg) that gives most likelihood increase (L^) if it is in the N-Node. 
At operation 407, a Y-Node to a N-Node class member transferring is noted. 

At operation 408, a determination is made if Ly > Tq. being a threshold value. 
If Ly is greater than Tp^at operation 410, ttie dtister Y^ is moved into its sister N-Node 
and saved on the N-Node. If is not greater tiian T^^at operation 412, tiie operation is 
to find one duster (N^ that gpives most likelihood increase {L^ if it is in the Y-Node. 
At operation 414/ a N-Node to a Y-Node dass itnember transferriiig is noted. 

At operation 416, a determination is made if Ly^ > T^, being a threshold value. 

If Lff^ is greater than T^, at operation 418, the duster N^is moved into the Y-Nodie and 
saved in the Y-Node and operation continues bade to operation 406. If Nj is not 
greater than T^^ then operation 400 continues at operation 420. 

At operation 420, a determiiiation is Dciade if Ly<Ti&I^<Tj, IfL^ isnotless 
than Tj or L„ is not less than T^, operation 400 continues back to operation 406. If Ly is 
less than & 1^ is less than T^, then operation 400 continues at operation 

At operation 422, at determination is made if a certain depth is reached for the 
decision tree. That is, the above operation is repeated at all levels of tiie dedsibn tree 
until a certain dept of the decision tree is reached. If a certain depth is not reached, 
operation 400 finds the next best node to spKt and continues back at operation 404. If a 
certain depth is readied, operation 400 ends. _ 
Fseudo-code for building an OML-DCSTree 

The following is ©cemplary pseudo-code for building an OML-DCSTree as 
described above with respect to FIG. 4. The pseudo-code can be implemented using 
known programming languages such as "C** or "C++." 

Let N = {nl, n2,...,/i^ } be flie node corresponding the "no-answer" for the best 
question Q^r Y = {yl, y2,..., y^^ ] tiie node corresponding the "yes-answer" to ttxe best 
question at its parent node, and M = Mf* + be the total number of seen teiphones at 
their parent node. 
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1 Initiate conventional decision tree (omitted). 

2 Reshiiffle the class members between Y/N-nodes based on the likelihood 
increase for Question Q,,. 

float prob^old = prob (N) + Prob (Y); 
float prob^best = prob_old; 

For(i=l,i<M^i++){ 

float prob^minus = prob (N+i); 
float prob_pliis = prob (Y-i); 

if ( (prob_mintis+prob_plus) > prob .best ) ( 
indexed; 

prob_best=^robjtninus+prob^lus; 
1 

} 

if( (probj^est - prob_old)>Certain criterion ){ 

Move ti\e triphone duster indexed by "index" cluster to N-node 
from the Y-node 
}else{ 
break; 

1 • 

The above procedure will repeated starting from N-Node; 

Then the above two-way dass membership reshtiffle is repeated until it 

meets some condition. 

3 Record the new class members for the two new Y/N-nodes. 

4 Repeat the procedure 2-3 until it reach certain depth of the DCStree. 

The decision tree can be trained in the conventioital way. 

FIG. 5 is an exemplary diagram of tree nodes in a decision tree 500. The 
decision tree 500 indudes a root node 502 for ihe word '"b** and leaf nodes 504 and 506. 
For one embodiment, leaf node 504 represents a "Yes" dusta: having triphones ba, be, 
bi, and etc. and leaf node 506 represents a "No" duster having triphones bo, bu, and 
etc. which was generated by a rule-based approach. The triphones ba, be, and bi of 
leaf node 504 may be moved to leaf node 506 based on a maximiim likelihood as 
explained in FIG. 4. 

FIG. 6 is an exemplary diagram of a maximum likelihood decision tree 600 to 
illustrate decoding using ihe maximum likelihood decision tree. FIG. 6 shows how to 
find the appropriate triphone dass (i.e., leaf node) via a ML decision tree during 
decoding or speech recognition. 
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Referring to FIG. 6^ tfie decision tree 600 includes a plurality of nodes 602 
through 618. Each of the nodes 602 through 618 correspond to one dass. There is 
node (i.e., node 602), which contains all the known allophones of this dass. For 
example, node 602 indudes the known allophones (tl, t2, t3, t4, t5, t6, t7, and tS), which 
is also the sum of the allophones for its two child or "yes-no" nodes 604 and 606. 
Node 604 relate to the allophones of (tl^ t2, t3, t4, t5) and node 606 relates to the 
allophones (t6, tT, t8). Node 608 relates to the allophones (tl, t2, t3)^ leaf node 616 
relates to the allophone (tl), and leaf node 618 relates to the allophone (t2, t3). Also, 
leaf node 610 relates to ftie allophones (t4, t5), leaf node 612 relates to die allophone (t6, 
17), and leaf node 614 relates to the allophone (t8). 

During a speedi recognition or decoding process, each triphone of an unknown 
speech signal must be matched with a suitable dass that corresponds to the triphone in 
the decision tree 600. For example, dxiring the speech recognition or decoding process, 
the process traverses the decision tree according to the dass membership information 
stored in each tree node, that is, if it belongs to the class of "yes" node, then it goes into 
"yes" node, otherwise, it goes into the "no" node. This process proceeds on xmtil it 
reaches a leaf node, where the final triphone class can be foimd* If this triphone has 
not been found in the dedsioii tree, it will be dealt with in traditional way of which the 
phonetic context based leave node dass is used. 

HG. 7 a flow chart iUustraling an operation 700 to decode speech usm 
maximiom likelihood (ML) tree according to one embodiment 

At operation 702, all the seen allophones of fiiis dass are saved on the ML 
decision tree root node (from training knowledge). At operation 704, the decision tree 
is traversed according to the dass membership information stored in each tree node, 
that is, if it belongs to the dass of "yes" node, then it goes into "yes" node, otherwise, 
it goes into the "no" node. 

Atoperation706,adeter]iuriationismadeif a leaf node has been reached. Ifa 
leaf node has been reached, operation 700 continues at operation 718 in which a final 
decision is determined about a triphone X belonging to whidi dass* If a leaf node has 
not been reached, operation 700 continues at operation 708. 

At operation 708, a determination is made if a triphone X bdongs to a Yes dass. 
If it does belong to flie Yes dass, operation 700 continues at operation 710. At 
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Operation 710, the operation goes into the Yes node and continues at operation 702. If 
it does not belong to the Yes dass, operation 700 continues at operation 71Z 

At operation 712, a determination is made if triphone X belongs to the No class. 
If it does belong to the No class, operation 700 continues at operation 714. At 
operation 714, the operation goes into the No node and continues at operation 702. If 
it does not belong to the No dass, operation 700 continues at operation 716. 

At operation 716, tiie operation selects the phonetic context based leave node 
and continues at operation 702. 

Thus, a method and system for generating and searching an optimal maximum 
likelihood decision tree for hidden markov model (HMM) based speech recognition 
have been described. In the foregoing specification, tiie invention has been described 
with reference to specific exemplary embodiments thereof. It will, however, be 
evident that various modifications and changes may be maide thereto without 
departing from the broader spirit and scope of the invention as set forth in the 
appended claims. The specification and drawings are, accordingly, to be regarded in 
ani illustrative sense ratiier than a restrictivie sense. 
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CLAIMS 

What is claimed is: 
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1. A speech processing method comprising:: . 
receiving speech signals; 

processing the received speech signals to generate a plurality of phoneme 
dusters; 

grouping the phoneme dusters into a first duster node and a second duster 
node; and 

determining automatically if a phoneme duster in the first duster node is to be 
moved into the seoond duster node based on a likelihood increase of the phone duster 
of th@ first duster node from being in the first duster node to being in the second 
duster node. 

2. The speech processing method of claim 1, further comprising: 

moving the phoneme duster in tiie first dusta: node into the second duster node if the 
first duster node is determined to be moved into the second duster node. 

3. The speedi processing method of daim 2^ wherein moving the phoneme duster 
in the first duster node into the second dxister node indudes: 

moving the first duster node into the second duster node if the most likelihood 
increase is more than a threshold value. 

4. The speech processing method of daim 1, wherein the phoneme dusters are 
triphone dusters based on a hidden markov modd (HMM). 

5. The method of daim 1, wherein the grouping of the phoneme dusters indud^: 
grouping the triphone dusters according to answers to best phonetic context 

based questions rdated to the triphone dusters. 
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6. A speech processing system comprising: 
an input to receive speech signals; 

a processing imit to process received speech signals, to generate a pluraKty of 
phoneme clusters from the processed received speech signals, to group the phoneme 
dusters into a first dtister node and a second duster node, and to determine 
automatically if a phoneme dxister in the first duster node is to be moved into ttie 
second duster node based on a likelihood increase of tiie phone duster of the first 
duster node from being in Ae first duster node to being in the second duster node. 

7. The speech processing system of daim 6, wherein the processing imit is to move 
the phoneme duster in the first dxister node into tiie second duster node if tiie first 
duster node is determined to be moved into the second duster node. 

8. The speech processing system of daim 7, wherein the processing unit is to move 
tiie first duster node into the second duster node if the most likelihood increase is 
more than a threshold value. 

9. The speech processing system of daim 6, wherein the phoneme dusters are 
triphone dustere based on a hidden ECiarkdv modd (HNINQ. 

10. The speech processing system of daim 9, wherein ftie processing unit is to group 
the triphone dusters according to answers to best phonetic a>ntext based questions 
rdated to the triphone dusters. 

11. A machine-readable medium that provides instructions, which if executed by a 
processor, cause the processor to perform the operations comprising: 

receiving speech signals; 

processing the received speech signals to generate a plurality of phoneme 
dusters; 

grouping the photieme dusters into a first duster node and a second duster 
node; and 
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detenziining automatically if a phoneme duster in die fir^t duster node is to be 
moved into the second duster node based on a likelihood increase of the phone duster 
of the first duster node from being in the first duster node to being in the second 
duster node. 

VL The machine-readable medium of claim 11, further providing iixstructions, 
which if executed by a processor, cause the processor to perform the operations 
comprising: 

moving the phoneme duster in the first duster node into the second duster node if the 
first duster node is determined to be moved into the second duster node. 

13. The machine-readable medium of claim H, further providing instructions, 
which if executed by a processor, cause the processor to perform the operations 
comprising: 

moving the first duster node into the second duster node if the most likelihood 
increase is more than a threshold value. 

14. The machine-readable meditun of daim 11, further providing iiis^ 
whidi if executed by a processor, cause the proces^r to perform the operations 
comprising: 

processing the received speech signals to generate a plurality of phoneme 
dusters that are triphone dusters based on a hidden markov model (HMM). 

15- The madiine-readable medium of daun 14, further providing instructions, 
which if executed by a processor, cause the processor to perform the operations 
comprising: 

grouping the triphone dusters according to answers to best phonetic context 
based questions related to Ihe triphone dusters. 
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